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20.  Ab=t*ias^  continued. 


In  addition,  it  is  shown  that  the  existence  of  a stabilizing  constant  feed- 
back gain  and  the  reliability  of  its  implementation  is  equivalent  to  the  con- 
vergence properties  of  a set  of  coupled  Riccati-like  matrix  difference  equations 
In  summary,  these  results  can  be  used  for  off-line  studies  relating  the 
open  loop  dynamics,  required  performance,  actuator  mean  time  to  failure, 
and  functional  or  identical  actuator  redundancy,  with  and  without  feedback  gain 
reconfiguration  strategies. 


iv.::  M 


UNCLASSIFIED 


SICUNITV  CLASSIFICATION  OF 


t(Whm%  Data  Snraratf) 


iL 


AFOSR.TR.  78-  1542 

(Proceeding  of  tree  Conference  on  Decision  and  Control,  San  Diego, 


LIDS -P-861 


1979.) 


oh  RELIABLE  CONTROL  SYSTEM  DESIGNS  WITH  AMD  WITHOUT  FEEDBACK  RECONFIGURATIONS  • 


J.  D.  Birdwell 
Farris  308 

Department  of  Electrical  Engineering 
Tha  Univarsity  of  Tannassaa 
Knoxville , TN  37916 


D.  A.  Csstanon  and  M.  Athans 
Roost  3S-308 

Laboratory  for  Information  and  Cacision  Systaas 
Massachusatts  Instituta  of  Technology 
Cambridge , MA  02139 


I 

I 

i 


Abstract 

This  paper  contains  an  overview  of  a theoretic- 
al  framework  for  tha  dasign  of  raliabla  multivariable 
control  systaas,  with  spatial  aaphasis  on  actuator 
failures  and  nacassary  actuator  redundancy  levels. 
Using  a linear  nodal  of  tha  systaa,  with  Markovian 
failure  probabilities  and  quadratic  perforaanca  in- 
dex, an  optlaal  stochastic  control  problaa  is  posad 
and  solved.  Tha  solution  requires  tha  iteration  of 
a sat  of  highly  coupled  Riccati-like  matrix  differ- 
ence aquations,  if  thase  converge  one  has  a reli- 
able design,  if  they  diverge,  tha  dasign  is  unre- 
liable , and  tha  systaa  dasign  cannot  bo  stabilised. 

In  addition,  it  is  shown  that  tha  existence  of 
a stabilising  constant  feedback  gain  and  tha  rail- 
ability  of  its  implementation  is  equivalent  to  tha 
convergence  properties  of  a set  of  coupled  Riccatl- 
lika  matrix  difference  equations. 

In  suaaary,  thase  results  can  be  used  for  off- 
line studies  relating  the  open  loop  dynamics , re- 
quired perforaanca,  actuator  aaan  tiaa  to  failure, 
and  functional  or  identical  actuator  redundancy, 
with  and  without  feedback  gain  reconfiguration 
strategies. 

1.  Introduction 

This  paper  is  an  overview  of  a research  effort 
which  addresses  sane  of  the  current  problems  in  inter- 
facing systaas  theory  and  reliability,  and  puts  this 
research  in  perspective  with  the  open  questions  in 
this  field.  Reliability  is  a relative  concept,  it 
is,  roughly,  the  probability  that  a systaa  will  per- 
form according  to  specifications  for  a given  Mount 
of  tine.  The  motivating  question  K-‘-<  — * this  report 
is.  What  constitutes  a raliabla  systaa? 

If  a theory  were  available  which  allowed  a com- 
parison between  alternate  designs,  based  on  both  tha 
expected  systaa  reliability  and  the  expected  systaa 
perforaanca,  it  would  greatly  sl^lify  the  current 
design  aethodology.  It  is  unfortunate  that  at  pre- 
sent there  is  no  accepted  aethodology  for  a daterain- 
ation  of  expected  systaa  perfosaaaoa  which  accounts 
for  changes  la  the  perforamace  characteristics  due 
to  failure,  repair  or  reconfiguration  of  systaa  func- 
tions. This  report  presents  such  a methodology  for 
a specific  class  of  linear  systaas  with  quadratic 
cost  criteria. 


•This  research  was  supported  by  tha  Fannie  and  John 
Herts  Foundation,  NASA  asms  grant  NGL-22-009-I24, 
and  ATOdR  grant  77-12B1.  The  work  was  performed 
while  J.  D.  Sir dwell  was  a graduate  student  at 

N.I.T. 


Basically,  the  reliability  of  a systaa  is  tha 
probability  that  the  system  will  perform  according 
to  specifications  for  a given  amount  of  time.  In 
a system-theoretic  context,  the  specification  which 
a systaa  must  neat  is  stability,  also,  since,  at 
least  for  most  mathematical  models  of  systems,  sta- 
bility is  a long-term  attribute  of  the  system,  the 
amount  of  time  for  which  the  systaa  must  remain 
stable  is  taken  to  be  infinite.  Therefore,  tha 
following  definitions  of  systaa  reliability  are 
used  in  this  paper. 

Definition  1,  A systaa  (implying  the  hardware  con- 
figuration, or  mathaoatical  modal  of  that  configu- 
ration, and  its  associated  control  and  estimation 
structure)  has  reliability  r where  r is  the  proba- 
bility that  the  systaa  will  be  stable  for  all  time. 

Definition  2,  A system  is  said  to  be  reliable  if 
r ■ 1. 

Definition  3,  A systaa  dasign,  or  configuration, 
is  reliable  if  it  is  stabilixable  with  probability 
one. 

These  definitions  of  reliability  depend  on  tha  de- 
finition of  stability,  and  for  systaas  which  can 
have  more  than  one  node  of  operation,  stability  is 
not  that  easy  to  determine.  In  this  paper,  stabi- 
lity will  mean  either  naan-square  stability  (over 
sons  randan  space  which  will  be  left  unspecified 
for  the  nonant) , or  cost-stability  (again,  an  ex- 
pectation over  a certain  randoa  space),  which  is 
basically  the  property  that  the  accumulated  cost 
of  systaa  operation  is  bounded  with  probability 
one.  (Tha  definition  of  cost  is  also  deferred.) 

Tha  reliability  of  a systaa  will  depend  on 
tha  reliabilities  of  its  various  exponents  and  on 
their  interconnections.  Thus,  the  systaas  engineer 
oust  have  an  understanding  of  tha  probabilistic  ne- 
chanisos  of  component  failure,  repair,  and  systaa 
reoonf iguration . 

Component  failures,  repairs,  and  reconfigura- 
tions are  nodsled  in  this  paper  by  a Markov  chain. 
Only  catastrophic  changes  la  tha  system  structure 
are  considered,  degradations  are  not  modeled.  The 
hasard  rate  is  assumed  to  be  constant,  resulting  in 
an  exponential  failure  distribution,  in  tha  dis- 
crete-time case,  to  which  this  paper  is  confined 
exclusively,  the  hazard  rate  becomes  tha  probabili- 
ty of  failure  (or  repair  or  reconfiguration)  be- 
tween tioe  t and  tine  tel. 

It  is  now  necessary  to  define  precisely  the 
nodes  of  operation  and  their  dynamic  transitions. 
The  terms  system  configuration  and  system  structure 
will  be  used. 
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Definition  4 1 System  Structure  ■ a possible  soda 
of  operation  for  a gi van  systaa i tha  components, 
thair  intareonnactiona , and  tha  inforaation  flow 
In  tha  systaa  at  a qi van  time. 

Dafinitlon  5<  System  Configuration i Tha  original 
design  of  tha  ayataa,  accounting  for  all  aodalad 
aodaa  of  o par at ion,  and  tha  Markov  chain  govarning 
tha  configuration,  or  atructural,  dynaalca  (transi- 
tiona  among  tha  various  structural) . In  thia  pa- 
par,  structural  ara  rafarancad  by  convention  by 
tha  aat  of  non-nagativa  integers 

I e {0,^,2, 3 L}  (1.1) 

An  important  question  In  reliability  is  tha 
offset  of  redundancy  on  ayataa  performance.  In 
other  words , how  should  tha  allocation  of  control 
resources  be  allocated  to  tha  redundant  components, 
and  how  should  tha  coaponent  reliabilities  affect 
tha  choice  of  an  optimal  control  law?  Tha  control 
methodologies  prasantad  in  thia  paper  answer  tha 
question  fer  a specific  class  ayataa  configurations. 
They  yield  a quantitative  analysis  of  tha  effec- 
tiveness of  a given  .ayataa  design,  whara  effective- 
ness is  a quantity  relating  both  tha  performance 
and  tha  reliability  of  a configuration  da sign. 

Previously,  several  authors  have  studied  the 
optimal  control  of  systaaa  with  randomly  varying 
structure.  Most  notable  aaong  these  is  Wonhaa  (1) , 
where  tha  solution  to  tha  continuous  tlaa  linear 
regulator  problem  with  randomly  jumping  parameters 
is  developed.  Thia  solution  is  similar  to  the  dis- 
crete tiaa  switching  gain  solution  prasantad  in 
Section  3.  Wonhaa  also  proves  an  existence  result 
for  tha  steady-state  optimal  solution  to  tha  con- 
trol of  systaaa  with  randosUy  varying  structural 
however,  tha  conclusion  is  only  sufficient i it  is 
not  necessary,  similar  results  ware  obtained  in 
Beard  (21  for  the  existence  of  a stabilising  gain, 
vhsre  tha  structures  were  of  a highly  specific  fora; 
these  results  were  necessary  and  sufficient  alge- 
braic conditions,  but  cannot  be  readily  generalised 
to  lass  specific  classes  of  systems.  Additional 
work  on  tha  control  problem  for  this  class  of  sys- 
taas  has  been  dona  by  Swordar  (3),  Rather  C Luen- 
bergar  (41 , Bar-whalaa  a Si van  (3] , Wlllner  (61 
and  Piaree  c Swordar  [7J . Tha  dual  problem  of 
stats  estimation  with  a system  with  randca  parame- 
ter variations  over  a finite  sat  was  studied  in 
Chang  c Athens  (81 . 

Recently,  tha  robustness  of  tha  linear  qua- 
dratic regulator  has  bean  studied  by  Wong,  at.  ai. 
[91  and  Safonov  t Athens  [101.  Section  6 of  thia 
paper  gives  necessary  and  sufficient  conditions 
for  tha  existence  of  a robust  linear  constant  gain 
control  law  for  a specific  class  of  systaaa. 

r Mas  of  tha  prelladnary  results  on  which  this 

research  was  based  ware  prasantad  in  unpub ligbaL' 
fora  at  tha  1977  Joint  Automatic  Control  Confar- 
anee  in  San  Pranciseo  by  Birdwall,  and  published 
for  tha  1977  USB  Conference  on  Decision  and  Con- 
trol Thgpry  in  New  Orleans  by  Birdwall  6 Athens 
[11] . {This  paper  is  based  on  tha  results  in 
Birdwall  (12|Tj 

There  e/e  two  major  contributions  of  this  re- 
iser ch.  first,  tha  classification  of  a system  de- 
sign as  reliable  or  unreliable  has  been  equated 
with  tha  existence  of  a steady-state  switching 
gain  and  coat  for  that  design.  If  this  gain  does 


not  exist,  than  tha  systaa  design  cannot  ba  stabil- 
ized i hence,  it  is  unreliable.  Tha  only  r scour sa 
in  such  a case  is  to  usa  more  reliable  components 
and/or  no  re  redundancy.  Reliability  of  a systaa 
design  can  therefore  ba  determined  by  a test  for  c 
convergence  of  the  sat  of  coupled  Xiccati-lika 
equation  as  tha  final  time  goes  to  infinity. 

The  second  major  contribution  lies  in  tha  ro- 
bustness implications.  Precisely,  a constant  gain 
for  a linear  feedback  control  law  for  a set  of  li- 
near systaaa  is  said  to  be  robust  if  that  gain 
stabilizes  each  linear  systaa  individually,  i.e., 
without  regard  to  tha  configuration  dynamics.  Tha 
problem  of  determining  whan  such  a gain  exists, 
and  of  finding  a robust  gain,  can  ba  formulated  in 
tha  context  of  this  research.  As  a result,  this 
methodology  gives  an  algorithm  for  determining  a 
robust  gain  for  a sat  of  linear  systaaa  which  is 
optimal  with  respect  to  a quadratic  cost  criterion. 
If  the  algorithm  does  not  converge,  then  no  robust 
gain  exists. 

for  tha  purpose  of  brevity,  seat  result  will 
be  stated  without  proof.  Tha  reader  aay  find  these 
proofs  in  reference  [12] , and  in  tha  papers 
currently  in  preparation. 

2.  Problem  Statement 

Consider  the  systaa 

*»l*&*t  * lk(t)2it  <a-1) 

where 

(2.2) 

(2.3) 

(2.4) 

and,  for  each  k,  an  elaaent  of  an  indexing  sat  I 
kt:>  (0,1,2,  . . . ,L)  (2.5) 

Bk  C Rn*"  (2.6) 

whara 

ike(ii>iei  (J*7> 

The  index  k(t)  is  a random  variable  taking  values  in 
I which  is  governed  by  a Markov  chain  and 

Ifl  ‘ ilt 


X t c r" 

u C R* 

— * n*  n 
A e R 


w t R 


L+l 


(2.8) 

(2.9) 


where  v.  . is  tha  probability  of  k(t)  - i,  given  no 

on-line  Information  about  k(t) , and  w . is  the  init- 
ial distribution  over  I. 

Zt  is  assumed  that  tha  following  sequence  of 
events  ooeurs  at  each  tiaa  ti 

1)  z la  observed  exactly 

2)  than  lk(t_u  switches  to  »k(t) 

3)  than  u is  appUad. 

Conaidar  tha  structure  sat  {b  k(I  indexed  by 
I.  Define  tha  structural  trajectory  ^ to  ba  a se- 
quence of  elements  k(t)  in  I which  select  a specific 
structure  >k(t)  at  tiaa  t, 

x_  - (k(0),k(l) k(T-l) ) (2.10) 

The  structural  trajectory  ^ is  a randca  variable 
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with  probability  of  occuranca  ganaratad  from  tha 
Markov  aquation  (3.8). 

p<v  ■ JL\(t).t  «-u> 

t-o 

wh*r«  thm  control  interval  is 

(0,1,2 T-1,T}  (2.12) 

for  tha  finita  tiaa  problam  with  terminal  time  T. 
Than  for  a givan  stata  and  control  trajactory 


T-1  • 

(x  »u*.  ) a.«n  by  (2.1)  and  x fro*  a *•- 

quanca  of  controls  (u  t , tha  coat  indax  la  to 
ba  tha  standard  quadratic  cost  criterion 
tux 

Jt(*t'  (-t  '-t  't-0  ■ itiit  + 

(2.13) 

Tha  objactiva  is  to  chooaa  a faadback  control 
law,  which  aay  dapand  on  any  past  intonation  about 
or  u , napping  x t into  u( 

♦*  : Rn  - r“  (2.14) 

«-15) 

such  that  tha  expected  valua  of  tha  cost  function 
^ from  aquation  (2.13) 

jt  " *l  At  I So1  «•“> 

' a 

is  minimised  ovar  all  poaaibla  mappings  ^ at  4^. 

3.  Tha  Optimal  Solution 


normally,  a control  law  of  tha  fora  (2. IS) 
must  provide  both  a control  and  an  estimation 
function  in  this  type  of  p rob 1 ami  hence  tha  label 
dual  control  is  usad.  Hare,  tha  structure  of  tha 
problem  allows  tha  asset  determination  of  k(t-l) 
from  s t , * for  almost  all  values  of  u ^ . 

This  result  is  stated  la  tha  following  Uma 
lea—  1 (121 t For  tha  sat  {8  ^ ^ £ where  tha 

Bk  's  are  distinct,  tha  sat 

(£k,ttl  ‘ lit  * lkit}  U *****  — * *» 
for  almost  all  vmluas  of  u . 

Ignoring  tha  sat  of  controls  of  maasura  zero 
for  which  tha  members  of 


&k.t+l* 


k*0 


(3.1) 


era  not  distinct,  than  for  falsest)  aay  control 
which  tha  optimal  algorithm  selects,  tha  resulting 
state  * can  ba  ccmparad  with  tha  members  of  tha 
sat  (3.i)  'tor  an  asset  match  (of  which  thara  is 
only  one  with  probability  1),  and  k(t)  is  identi- 
fied as  tha  generator  of  that  matching  member 


Ifc.tal * 

Since  perfect  Identification  is  tha  bast  any 


algorithm  can  achieve,  tha  optimal  control  law 
u*  ■ a*(s  > can  ba  calculated  with  tha  aasuapticn 


that  k(t-l)  is  known,  since  thia  is  tha  case  with 
probability  an>..  Thus,  this  solution  will  ba  la- 
beled tha  switc  line  gain  solution,  since,  for  each 
tine  t,  L+l  optimal  solutions  are  calculated  aprlorl . 
ind  ana  solution  is  chosen  on-line  for  each  time  t, 
based  on  tha  past  measurements  x and  u , 

which  yield  perfect  knowledge  of  k(t-l). 

Dynamic  programing  is  usad  to  derive  tha  op- 
timal switching  gain  solution.  It  has  baan  proved 
(12)  that  at  each  tlma  t,  tha  optimal  expected 
cost-to-go,  givan  tha  systam  structure  k(t-l),  is 


V (xt  ,k(t-l)  ,t)  - x‘  Sk  e£e 


(3.2) 


whara  tha  S.  are  determined  by  a sat  of  L+l 
coupled  Ricc*tl-like  equations  (ana  for  each  possi- 
ble configuration) ■ 


Pik  — i , t+1 


• [X*a*I  tu«]|  *** 

The  optimal  control,  given  k(t-l)  - k,  is 

T 

(LPik  -iii,t+l-*t 


-1 


(3.3) 


i-0 

Writing 

u • G X 
-k,t  -k.tit 

then 


(3.4) 


(3.3) 


* pi)t  &i  4i,t+i^ 

Thus,  u ■ ♦ (x  ) is  a switching  gain  linear  con- 
trol law  whlehahoenda  on  k(t-l).  Tha  variable 


(3.6) 


k(t-l)  is  determined  by 
k(t-l)  ■ i iff  x t • Rxt_j+ 


(3.7) 


Mote  that  the  8 . *a  and  tha  optimal  gains 
8k(tcaa  ba  computed  ’ off-line  and  stored.  Than 

at  each  elms  t,  tha  proper  gain  is  selected  on- 
line from  k(t-l),  using  aquation  (3.7),  as  in 
Figure  1. 

Thia  solution  la  quite  complex  relative  to 
tha  structure  of  tha  usual  linear  quadratic  solu- 
tion. Bach  of  tha  Riceati-lika  aquations  (3.7) 
involves  tha  same  complexity  as  the  Rlceati.  aqua- 
tion for  the  linear  quadratic  solution.  In  addi- 
tion, thnre  is  tha  on- line  complexity  arising 
from  tha  lap lamentation  of  gain  scheduling . in 
Section  4,  a non-switching  gain  solution  will  ba 
presented  which  baa  an  identical  on-line  struc- 


tor*  to  that  of  tha  linear  quadratic  solution,  but 
has  similar  off-liaa  computational  complexity  to 
that  of  tha  switching  gain  solution.  This  non- 
switching  gain  solution  is  suboptiaal,  but  requires 
less  on-line  caoplexity. 


Figure  1:  The  switching  gain  control  law. 


A steady-state  solution  to  equation  (3.7)  may 
exist,  but  the  conditions  for  its  existence  are 
unknown.  The  steady-state  solution  would  haws  the 
advantage  that  only  one  set  of  gains  need  be  stored 
on-line,  instead  of  requiring  a set  of  gains  to  be 
stored  for  each  tiae  t.  Since  the  steady-state 
solution  is  s imply  the  value  to  which  equation  (3.7) 
converges  as  it  is  iterated  backward  in  tiae,  at 
present,  the  equations  can  be  iterated  numerically 
until  either  they  converge  or  neat  some  test  of 
non -convergence . The  possibility  of  limit  cycle 
solutions  in  the  switching  gain  computations  is 
excluded  by  the  following  laaui 

ta— a 2 (121 1 If  the  optimal  expected  cost-to-go 
at  tiae  t is  bounded  for  all  t,  then  equation  (3.7) 
converges. 

The  existence  of  a steady-state  solution  to 
the  switching  gain  problaa  establishes  a division 
of  system  designs  into  those  which  are  inherently 
reliable  and  those  which  are  unreliable.  Bvua 
though  conditions  to  test  for  the  trl  at eons  of  the 
steady-state  solution  are  unavailable,  software 
can  be  used  with  iteration  for  tha  test. 

4.  The  Son-switching  sain  Solution 

In  this  Section,  the  peraisslble  controls  are 
restricted  to  be  of  the  linear  feedback  form 

at“  2t«t  (4.1) 

where  the  gain  natrix  0 is  restricted  to  be  a 
function  only  of  tima  and  tha  Initial  conditional 
l.a. , it  cannot  4 a pend  on  x or  it  . Tha  objective 
is  to  minimise  over  the  set  of  admissible  controls 
the  expectation  of  (2.13)  whara  tha  axpectatlon  la 
taken  over  the  set  of  poesible  structural  trajec- 
torlas 

>,<{  > (4.2) 


and  tha  sat  of  Initial  conditions  x . Tha  re- 
sulting control  law  will  be  a non-si/ltching  law, 
using  gains  determined  aprlorl.  t . 

Thus,  the  optima  1~ control  law  ufc»  £.t£t 

should  minimize  the  cost 


* * I0 


- E 


T-l 


t*0 


(4.3) 


over  the  set  of  admissible  controls. 

Sines  the  structure  of  u • 0 x is  fixed, 
the  problem  Is  equivalent  to  mlnimfkin£>  In  an 
open- loop  sense,  the  cost  function 

■ B[is*as*4a*-tS£ejtt 


(4.4) 


with  respect  to  the  gain  natrix  G , teO,l,...,T-l. 

The  matrix  ntniswia  principle  ot  Athens (13)  la 
used  to  determine  the  necessary  conditions  for  tha 
existence  of  u*  (or  equivalently,  G*  ).  Let  tha 
initial  state~jr  be  a zero  mean  random  variable 
which  is  independent  of  any  structure.  Let 

' “fcoiollol  ' *'i0£o> 


(4.5) 


be  the  covariance  natrix  of  x.  . 

Defining  the  covariance  of  x~°  ea 

Ie  A ll0' 

and  if  we  define 

5.j.,t  " *l-t-t  lk(t"l,“t'Io 1 

than  the  natrix  can  be  defined  recursively  as 


(4.6) 


(4.7) 


and  tha  relation 


for  t > 1. 
T 


(4.6) 

(4.6) 


it*  iV  l 


i-0  t-l 


l.t 


t >0 


is  obvious  fr 


direct  ealculaticn. 


(4.10) 


lyccM 

(4.6)  I 


At  this  stage,  an  equivalent  deterministic 
'Problem  All  will  be  defined  with  state 

Uq  for  t>0  and  state  £.  at  t*0.  The 
dynamics  are  than  defined  by  aquations 
md  (4.9). 


Definition  (Problen  AX) i For  the  systan  with 
antrix  state  (£  )*j  for  t > 0 and  for  t-o 

with  dynamical  dqUhtXooa  (4.6)  and  (4.9)  and 
antrix  control  g , minimize  tha  equivalent  deter- 
ministic cost  mi  (G  „ ) £*  i 


ZmKi.*0  [ X 


Vxi  2 + £'i2t 


♦ £T2iTl£0  *£0 


- [ . <a + «! *£ji  * trtr.ai  («.ii) 

t-0  c ' ' _T 

Note  that  since  the  expectation  In  aquation  (4.4) 
ia  ovar  all  structural  trajectories  x and  tha  ini- 
tial x Q also, 


Tha  symbol  J will  ba  uaad  exclusively  in  tha  fu- 
ture . Tha  one- stags , or  Instantaneous,  coat  at 
time  t is 

Jt“  ttlit  (a  * St^t11  <4.13) 

Problem  it  ia  completely  deterministic  in  tha 

‘ii.t’i-o  ' £o  “d  cootr81  £t  • 

At  this  point,  tha  minimi ration  ia  decomposed 
into  tuo  parts  using  tha  Principle  of  Optimality 
(141 . Tha  first  minimization  ia  ovar  tha  interval 
(1,2, . . . ,T-l} , and  for  thia  tha  matrix  ■»<»<■» 
principle  will  ba  used.  Tha  resulting  solution 
depends  in  general  on  tha  choice  of  5.  and  on  tha 

initial  conditions  £_  and  ir„  . “° 

# »o  —0 

Let  V (G.  ) ba  tha  optimal  coat  resulting 
from  tha  use  or  0 Q and  tha  optimal  sequence  G * , 

Sj  »•  • • * for  the  interval  (l,2,...,T>. 

The  second  minimization  is  than  over  G „ of  tha  ooat 

—0 

jt"  tet-o  (£*  fio-So’1  * v*  ‘So1  <4'14) 

Tha  Principle  of  optimality  states  that  thasa 
two  minimizations  result  in  tha  minimising  saquanca 

<C,t  for  Problem  AZ. 

Prom  [Athens , 13] , tha  Hamiltonian  for  tha  min- 
imization ovar  (l,2, . . . ,T-l}  is 

4-l.t’i-O'  ‘ij.ttl’j-O'  £t) 

i , T 

■ i *,  L,  tta  + ll*  2.+  ' 

1-0  Vi  l,t  t-  t 

♦ «*  I £ (*7“  l »1*1  . 

Lj-o\  \ i-o  jl  ‘■t-l  ^ * t,e 

_ P * e 


wharn  **•  °9»tata  matrix  is  <£  x 1“^. 

Prom  tha  nacassary  condition  for  tha  costata, 

4 i*.t  " H77  <«•!«> 

thm  propagation  of  S backward  in  tiaa  ia  darivad. 

*#h 


for  t C (l,2,3,...,  T-l)  (4. IS) 

, nabmim  /a  1 


♦ X pJi-^  -T5.j,t+i— t4j4j,t+i4j4t 
♦iVtaii^fiUUj.tPia!  (4-17> 

This  aquation  is  well-defined  for  any  saquanca 
, , T-l 

IG  > and  t > 0.  Tha  coat  V of  using  this  ar- 
bitrary saquanca  ovar  the  interval  (l,2 T}  is 

given  by 


*«2e>~>-tr  j ^ 41,1£ifl 


40  * j (*t1£0,T4i.i‘4«t20)  ♦ 2*£q * 2C 


•al 

Than  tha  cost  of  a given  saquanca  (G  )*  * of 
length  T ovar  the  interval  10, 1, . . .7  TKTs 

JT  - t*(£oi0  (£0  '£1 £t-1  )]  (4'J0> 

From  tha  Hamiltonian  minimisation  nacassary  condi- 
tion 


tha  following  relation  between  £ , „,S  . , and 

Gt  is  obtained.  *•*  “J'**1 

0 m a a J 1 r 

a “£t  iio  ^-1  “l-* 

*X"\ 

X "i1  Vi  £i'e]  (4-2J) 

Pmmsrki  At  this  point,  a two-point  boundary  value 
problem  has  bean  defined  with  tha  constraint  (4.22) 
relating  aquations  (4.17)  and  (4.4).  Equation 
(4.22)  is  not  explicitly  solvable  far  G because 
£ ^ t cannot  be  factored  out  of  the  sum  over  j , thus 

it  cannot  ba  usad  as  a substitution  rula  in  tha 
other  two  aquations.  At  this  time,  tha  solution 
of  a appears  intractable.  Thus,  although  nacas- 
sary Vwwlltlons  for  the  existence  of  G * , tha 
minimising  gain,  have  bean  established,  they  do 
not  readily  allow  for  the  solution  of  G*  , and 
certainly  do  not  admit  a closed-form  expression. 

S.  Steady-State  Non-Switching  Gaia  Solutions 

In  thia  Section  a modified  version  of  Problem 
A is  solved  which  yields  a computational  methodo- 
logy for  computing  tha  optimal  steady-state  non- 
switching gain  solutions.  It  will  be  established 
that  the  solution  to  this  modified  problem  con- 
verges to  tha  sans  limit  as  the  problM  in  tha 
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last  Section. 

Definition  61  (Stability)  G la  a constant 
stabilizing  gain  if  and  only  if  tha  rasultlng  system 

la  aaan-squara  stablai 

E(x  a * 1 ♦ 0 uf 

Oaflnltlon  7;  (Cost-Stability)  Tha  ayataa  (5.1) 
is  coat-atabla  if  and  only  if 

X &«&&«+ < “ 

with  probability  ana. 

Tha  inf inita-t-laa  problem  is  dafinsd  as  a 
■lnlal  ration  of 

J - lim  (5.2) 


whars  J is  tha  cost  function  for  tha  correspond- 
ing finite-time  problem.  Tha  sequence  which 
solves^  ttys  infinite-tins  versions  of  Problem  AZ 
is  (G  „ ) whan  a solution  exists.  A solution 
will  eki£tuif  there  exists  a sequence  of  gains 
for  which  tha  limit  in  equation  (5.2)  exists. 

This  definition  of  tha  infinite- time  problem  is 
chosen  rather  than  tha  definition  requiring  a 
minimization  of  tha  average  cost  par  unit  time 


which  are  tha  limit  of  these  aquations,  given  that 
tha  limiting  solution  Z , and  G exist,  where  it 
satisfias  ~3,e 


lim  it  t « ir 


Tha  following  Theorem  yields  an  explicit 
procedure  for  the  calculation  of  tha  steady-state 
non-switching  control  law. 

«* 

Th«or«at  Dsfin*  ths  s«qu«nc«  > fceO  ** 

following  equations i 


±\  i 


for  a given  terminal  time  T and 
G - lim  G (T) 

*TIS. 


^k.t(T) 


because  there  is  a direct  correlation  between  tbs 
boundedness  of  J over  all  T for  a constant  se- 
quence of  gains  ? and  mean  square  stability  of 
tha  system  (S.l)T 

Tha  concepts  of  stability,  cost-stability, 
and  axistanca  of  a steady-state  solution  are 
related  by  tha  following  lamsas  (12] i 

w 

Leamm  3i  A constant  sequanca  of  gains  (G) - is 
mean-square  stabilizing  if  and  only  if  there  esists 
a bound  •<•  such  that  J < 8 V T. 

^ m 

Lemma  4i  Any  sequanca  (G  ) ..  cost-stabilizas 
(5.1)  (with  probability  anb)c”°if  and  only  if  J<“. 

Tha  staady-stata  solution  for  Problem  AX  is 
defined  as  the  limiting  solution  to  aquations  (4.8) 
(4.17)  and  (4.22)  at  time  t,  first  as  and  than 
as  t*«  , if  this  limit  exists.  Tha  staady-stata 
values  for  a,  S . , and  , whan  they  exist,  sa- 
tisfy tha  following  equations i 

x L 

m "j  io  pji  \ a>  £i  <*•«> 

I,  - [ 2 ♦ 1 ♦ X pjt  * 

* * QT>j8jAl  (5.5) 


Llo  *4  M <5*#> 


V^.fl  * + ^,ttl  S#nst 

* * s5.t4*j.tbi  *Ast  > for  k£  1 

s^  (T)  - Q for  he  I (5.12) 

(The  parameter  (T)  is  suppressed  on  tha  right  hand 
side  of  aquations  (5.9  ) and  (5.11).] 

Than  tha  following  statements  are  equivalent. 

1)  Tha  gain  sequanca  (G  ) „ cost-stabilizas 

*”  ni.  t*o 

♦!„(„»  t 

2)  ||  lim  J ir  V ||  < - 

k-0  1 K,c 

3)  A cost-stabilising  gain  sequence  exists. 

4)  Tha  solution  to  Problem  A,  (G  )**  is  cost- 
stabilizing. 

in  addition,  if 

Sne  - for  all  t (5.13) 

• * 

£ (steady-stats)  Mists 
than 

—ns  m& 

Tha  proof  can  ba  found  in  airdwsll  (12] . The 
derivation  of  aquations  (5.91,(5.11)  and  (5.12) 
can  bo  found  in  lirdwall  and  A than  (11]  and  BirdweH 
[12] . A forthcoming  paper  will  contain  tha  complete 
theorem  and  proof.  Equation  (5.9)  to  (5.12)  will 
hereafter  ba  raferad  to  as  tha  solutions  to  Problem 
B,  which  is  described  in  (12]  and  is  omitted  hare 
dua  to  lack  of  space.  The  results  of  this  theorem 
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give  • direct  computational  procedur j for  calcula- 
ting the  optimal  steady  stats  gain  G aa  the  Halt 
of  gains  G Thera  are  seme  questions  as  to  the 

possibility1^  limit  cycles  on  the  calculation  of 
G However,  thj  theorem  guarantees  cost-atabi- 

L Of/  using  ) t-Q  whenever  the  system  Is 

cost-s  tabilizabln. 

6.  Robustness 

1 The  original  problem  (Problem  A)  can  be  for- 
mulated in  such  a way  that  the  sequence  (G  ) . , 

— nsfc  t*o 

will  cost-stabiliza  a set  of  linear  systems  with 
different  actuator  structures  Individually  whenever 
such  a stabilising  or  robust  gain  exists. 

Definition  8i  A gain  G Is  robust  if 

it+1-  <*  + “■<=>£ t <«-l> 

Is  stable  for  all  It.  This  is  the  same  as  requiring 
the  matrix  ( A+£ . G)  to  have  eigenvalues  inside  the 
unit  circle  for  all  k. 


e {0,1 ,?} 

The  cost  to  be  minimized  is 

m 

J “ E I it-  S J I 

t-0 


The  matrices  are  given  by 
[2. 71828  O.ol 


'1.71828  1.71828 

8 . ■ 

[-.63212  .63212 

0.0  1.71828' 

»,•  ». 

A 0.0  .63212  4 


1.  0.  0. 

0.  1.  0. 

0.  0.  1. 


1.71828  0.0 

-.63212  0.0 


For  these  matrices,  aquations  (5.9)  and  (5.11) 
converge,  giving  the  following  results 1 


For  tha  set  of  l+l  systems 

6 • 

■-1 

.089 

-.008413 

*k*t 

(6.2) 

— n u 

-1 

.028 

-.01444 

l 

112.8 

8.992 

(6.3) 

l Vi 

• 

(6,4) 

i-0  1 1 

.8.992 

6.835 

if  a robust  gain  exists,  than  (G  ) . _ is  a 

stabilizing  sequence  for  (6.1)  for  each  k,  and  if 
the  gains  G (T)  converge,  than  G is  a robust 
gain.  -“t  -n* 

Discussion;  With  Corollary  1,  a specific  existence 
problem  for  robust  linear  gains  is  solved.  Exis- 
tence of  a robust  gain  is  made  equivalent  to  tha 
existence  of  a finite  cost  infinite-time  solution 
to  Problem  B,  which  is  readily  computable  from 
equations  (5.9)  and  (5.11). 

Consider  tha  system  whose  transitions  are 
shown  in  Figure  2.  The  configuration  dynamics  are 
modeled  aa  being  in  any  structural  stats  with  equal 
probability  of  occuranca  initially  and  remaining 
in  that  state  forever 1 this  model  is  illustrated 
graphically  in  Figure  2 below. 

FtlMAWOM 


(HV  (££)'»•' 

Figure  2;  Harkov  transition  probabilities  for 
the  Example. 

The  state  dynamics  are 

*t“  l*l,t  *2,t'T 


A brief  check  will  verify  that  this  is  indeed  a 
robust  gain.  The  Riccati  solutions  for  this  problem 

are 


109.8 

9.0301  [ll4.3 

6.285 

£1“ 

9.030 

6.821J  1 [6.285 

6.836 

114.4 

11.66 

.11.66 

6.849 

The  non-switching  solution  converges  for  this 
system,  and  the  three  resulting  configurations  are 
stabilized.  Therefore  G is  a robust  gain.  Had 
the  solution  not  convergimr,  by  Corollary  1,  no  ro- 
bust gain  would  exist.  The  aprlorl  expected  cost 
(before  the  configuration  state  is  known)  is,  given 
£1 


J - xTCx 


7.  Conclusion 


In  conclusion,  the  unifying  concept  of  this 
report  is;  What  conatitutee  a reliable  control 
system,  or  a reliable  design7  A major  connection 
was  established  in  this  research  between  the  con- 
cepts of  reliability  and  stabi Usability,  itera- 
tive procedures  ware  developed  for  the  determina- 
tion of  whether  or  not  a given  linear  system  of  tha 
type  considered  la  this  report  is  reliable,  with 
respect  to  both  non-switching  and  switching  gain 
controllers.  A system  design  is  reliable  if  and 
only  if  the  set  of  coupled  Riccati-like  matrix 
difference  equations  for  the  switching  gain  solu- 
tion converges.  In  addition,  if  tha  matrix  differ- 
ence equations  converge  for  the  non-switching  gain 
solution,  then  tha  non-switching  control  law  yields 
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a robust  system  if  they  diverge,  no  robust  gain 
exists. 

Thia  paper  ia  an  overview  of  tha  raaulta  in 
Birdwall  [12] . Two  papar*  in  praparation  will  con- 
tain tha  proof*  of  tha  raaulta  which  ara  statad 
bar*. 
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